Method and device for detecting the orientation of an image

ABSTRACT

The invention relates to a method and a device for detecting the orientation of an image in a set of images. All images in this set of images contain at least one similar object. The method proposes to choose one image in this set of images as being a reference image. The orientation of the other images is detected based on the orientation of this reference image.

The invention relates to a device and a process for detecting theorientation of an image in a set of images.

This application claims the benefit, under 35 U.S.C. §365 ofInternational Application PCT/EP05/050022, filed Jan. 4, 2005, which waspublished in accordance with PCT Article 21(2) on Jul. 28, 2005 inEnglish and which claims the benefit of French patent applications No.0400067, filed Jan. 6, 2004.

The invention relates more generally to the automatic detection of theorientation of an image that may possibly also contain text.

Like scanners, digital cameras generate photos which are often viewed ona screen, such as a computer screen. These images may be viewedcorrectly, that is to say in the right orientation or in a sense thatdoes not allow the user to view them without having to turn his head tothe left or to the right, or even to lower his head.

Specifically, sensing apparatus, such as digital cameras, can captureimages in several senses, and in particular it is not uncommon for auser to take some of his photos in portrait mode and others in landscapemode. The photos thus taken are then transferred to a computer and allviewed in the same sense. Some will therefore be correctly viewed whileothers will require a 90, 180 or 270 degree rotation to be viewedcorrectly.

Certain sensing devices and in particular certain digital cameraspossess orientation sensors that detect a rotation of the objective andtransmit a rotation cue obtained from the sensor with the image. Thisallows the viewing device, by virtue of the rotation cue, toautomatically perform a rotation of the image so that the latter appearsin the correct sense.

Other devices use methods for extracting the low-level or high-levelcharacteristics of the image. This makes it possible to analyse thecontent of the image in terms of colour, texture and also in terms ofsemantic content.

However, such devices are not robust for all types of images.

The invention relates more particularly to the automatic rotation ofimages in viewing devices not receiving any orientation cue from thesensing device and based on the orientation of other images.

Accordingly, the invention relates to a method for detecting theorientation of an image in a set of images, comprising the steps of:

-   -   choosing a reference image from among the set of images,    -   detecting the orientation of said image as a function of the        orientation of the said reference image.

It frequently happens that the user takes several images representingthe same scene, certain images being taken in portrait mode and othersin landscape mode. The detection of a reference image from among theseimages may make it possible to detect the orientation of the otherimages representative of the same scene or of a similar-content scene.This happens especially frequently in respect of images representing aperson in a landscape, the user wishing in certain images to photographthe landscape and in others to make the person stand out by taking aclose-up shot.

The invention will be better understood and illustrated by means ofadvantageous, wholly nonlimiting exemplary embodiments and modes ofimplementation, with reference to the appended drawings, in which:

FIGS. 1 a and 1 b represent examples of decomposition into subimages,

FIG. 2 represents a decomposition of the subimages into blocks,

As indicated in FIGS. 1 a and 1 b, in the target image whose orientationis sought as well as in a reference subimage is positioned a subimage.

According to FIGS. 1 a and 1 b, the subimages are of smaller size thanthe image in which it is positioned. It is also possible for thesubimages to be identical to the images.

The two subimages have the same relative size with respect to the imagein which they are positioned. The two subimages have the samewidth/height ratio.

In the case where one or both subimages are smaller than the respectiveimage in which they lie, the subimage is positioned in such a way as toobtain a minimum distance between the subimage of the reference imageand the subimage of the target image whose orientation one wishes todetect. To this end, several positions of the subimages are tested andthe best position is retained.

In another embodiment, the subimage is positioned at the centre of theimage for the reference image and for the target image.

In another embodiment, the subimage is positioned according to a fixedrule depending on the class of the image. This presupposes that theimage is classed previously according to a type which may for example bethe type of scene (town, landscape, interior scene, person, etc.). It ispossible to define a rule as follows: for landscape and interior scenes,the subimage is placed in the centre of the image; for town scenes, thesubimage is placed at the bottom of the image.

In yet another embodiment, the subimage is placed according to a fixedrule depending on the presence and on the position of objects in theimage. This involves the prior detection of objects in the image. Objectcan also include face. The rule possibly then being to centre thesubimage on the object.

When the subimages are positioned in the reference image and in thetarget image, four measurements of visual distance are performed. Thefour measurements are performed:

-   -   between the subimage of the target image and the subimage of the        reference image,    -   between the subimage of the target image and the subimage of the        reference image having undergone a rotation of 90 degrees,    -   between the subimage of the target image and the subimage of the        reference image having undergone a rotation of 180 degrees,    -   between the subimage of the target image and the subimage of the        reference image having undergone a rotation of 270 degrees,

To do this, the subimages of the reference image and of the target imageare sliced up into blocks A*B, A vertically and B horizontally. FIG. 2illustrates an example of the slicing into blocks, where A and B areequal to 2.

This slicing is aimed at complying with the non-stationarity of theimages processed (the various zones of the images exhibit very differentcontents). The higher the number of blocks, the more this effect will betaken into account.

The number A*B should not however exceed a certain threshold dependingon the resolution of the images in order to contain enough information.

The four measurements of distance D between the target image andreference image are calculated thus:

$D = {\sum\limits_{i = 0}^{{({A^{*}B})} - 1}{w_{i}D_{i}}}$

The distances D_(i) represent the four distances D0, D1, D2, D3 betweenthe four blocks of the images, as indicated in FIG. 2.

A weighting factor w_(i) is applied for each block. The weighting candepend either on a classification of the blocks according to theirsignificance or on a measure of confidence in respect of the calculationof the distances.

Each distance D_(i) is calculated from M particular distances d_(m)according to the following formula:

$D_{i} = {\sum\limits_{m = 0}^{M - 1}{v_{m}d_{m}}}$

m representing the number of descriptors used for the calculation of adistance. The descriptors possibly being colour, texture for example.

The calculation of the distance d_(m) is carried out using processesknown to the person skilled in the art. It is possible to effect colourhistogram differences, contour histogram differences, the difference ofsubband energies, the difference of responses of directional filters soas to measure these distances.

A weighting factor v_(m) is applied for each distance d_(m) so as totake account of the confidence of calculation of the m distances.

The reference images are images whose orientation is known a priori. Theorientation of these images may be obtained through various methods, andin particular by detecting for example objects in these images or faces,but it is also possible to index them manually by the user. It is alsopossible to use methods different from the method proposed by thepresent invention making it possible to detect the orientation of animage and these methods may be used to detect the orientation of thereference image. Such a method is described for example in EuropeanPatent Application 1107182 filed in the name of the company EastmanKodak Company on 17 Nov. 2000 and entitled “determining orientation ofimages containing blue sky”.

When several reference images are available to the method, the choice ofthe reference image is performed by making a calculation of J distancesE_(j) between the given image and the reference images available to themethod.

Each distance E_(j) is calculated from L particular distances e_(l)according to the following formula:

${Ej} = {\sum\limits_{l = 0}^{L - 1}{q_{l}e_{l}}}$

A weighting factor q_(l) is applied to each distance e_(l) calculated.

Each distance e_(l) calculated is calculated according to knownprocesses, as indicated previously in respect of the distances d_(m).

The reference image for which the distance with the target image is thesmallest is then chosen.

Subsequently the measurements of distances D_(i) such as indicatedpreviously are calculated as is D.

D is therefore calculated between the target image and the referenceimage in four positions (turned by 0, 90, 180 or 270 degrees).

The smallest distance indicates the rotation that the target image mustundergo in order to be viewed in the correct sense. For example, if thesmallest distance is that between the target image and the referenceimage turned by 270 degrees, then the target image must be turned by 207degrees so as to be viewed in the correct sense.

1. Method for detecting the orientation of images in a set of capturedimages representing a similar scene, all the images in said set ofcaptured images containing at least one similar object, wherein themethod comprises the steps of: choosing at least one reference imagefrom the set of captured images, the reference image having anorientation that is known a priori, wherein said choosing comprisesselecting, for each target image in said set of captured images, asingle reference image among a plurality of reference images whoseorientations are known a priori, wherein said selected reference imageis the reference image that has a minimum distance to said target imageamong the plurality reference images; and detecting, by a hardwareprocessor, orientation of said target image as a function of theorientation of said selected reference image.
 2. Method according toclaim 1, comprising a step of calculating a visual distance between thereference image and the at least one other image.
 3. Method according toclaim 2, comprising a step of calculating the visual distance betweenthe at least one other image and the reference image for differentorientations of the reference image, wherein the different orientationsinclude the at least one other image and the reference image beingprovided in a first orientation, the reference image having undergone arotation of 90 degrees, 180 degrees, and 270 degrees with respect to thefirst orientation.
 4. Method according to claim 3, comprising a step ofdetermining a subimage in the reference image and a subimage in the atleast one other image, the calculation of the visual distance betweenthe at least one other image and the reference image being performed onthe respective subimages.
 5. Method according to claim 4, wherein saidsubimages have the same size.
 6. Method according to claim 4, whereinsaid subimages are centered with respect to the images in which they arepositioned.
 7. Method according to claim 4, wherein said subimages arepositioned in such a way that the visual distance between said subimagesare minimal.
 8. Method according to claim 4, wherein said subimages havea same width to height ratio.